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A PARALLEL JACOBSON -OKSMAN OPTIMIZATION ALGORITHM 


Terry A. Straeter and Athena T. Markos 
Langley Research Center 

SUMMARY 

A gradient -dependent optimization technique which exploits the vector -streaming 
or parallel -computing capabilities of some modern computers is presented. The algo- 
rithm, derived by assuming that the function to be minimized is homogeneous, is a modi- 
fication of the Jacobson-Oksman serial minimization method. 

In addition to describing the algorithm, conditions insuring the convergence of the 
iterates of the algorithm and the results of numerical experiments on a group of sample 
test functions are presented. The results of these experiments indicate that this algo- 
rithm will solve optimization problems in less computing time than conventional serial 
methods on machines having vector -streaming or parallel -computing capabilities. 

INTRODUCTION 

In recent years gradient -dependent optimization techniques have been used to solve 
a number of aerospace and aeronautical engineering problems with the digital computer. 
The most widely used gradient -dependent techniques are the conjugate gradient (CG) and 
Davidon-Fletcher -Powell (DFP) methods (see refs. 1 to 3). These methods are similar 
in many ways. The similarity of interest here is the fact that the methods are serial in 
nature. That is, the n -dimensional minimization is replaced with a sequence of univariate 
minimizations where the new direction is determined as a result of the value of the gradi- 
ent of the function and the previous univariate minimization. When these methods were 
developed, their serial nature did not seem to be a particular disadvantage. In fact, many 
felt it an advantage since the digital computers for which the algorithms were programed 
were also serial. However, a new generation of computers with vector -streaming or 
parallel -computing capabilities has appeared (for example, Uliac IV, C. mmP, and STAR). 
These capabilities are used here to synchronously or simultaneously evaluate the function 
and its gradient at a number of different points. These advanced computing capabilities 
cannot be exploited by the CG and DFP methods. 

Thus far, some work has been done to develop optimization algorithms which exploit 
these parallel capabilities. Avriel, Karp, and their associates (refs. 4 and 5) have worked 
on the univariate minimization problem. Miranker et al., have developed a nongradient 



"parallel" version of the Powell -Zangwill method (refs. 6 to 8). Straeter (ref. 9) has 
developed a parallel variable metric (PVM) technique which is gradient dependent and a 
close relative of the Davidon-Broyden (refs. 10 to 12) rank one techniques. Although the 
PVM, CG, and DFP differ in that the former exploits parallel -computing capabilities and 
the others do not, they are similar in that they are all closely tied to quadratic functions 
because the model function used to develop the algorithms is a quadratic. Algorithms like 
PVM, DFP, and CG which minimize positive definite quadratics in a finite number of steps 
exhibit good convergence properties on more general functions because near its minimum 
a sufficiently smooth function is closely approximated by a quadratic. However, this 
quadratic may be singular. In such cases, the convergence of these algorithms is slowed 
considerably. 

Recently, Jacobson and Oksman (ref. 13) have proposed an algorithm (denoted by JO) 
which assumes that the model function is homogeneous. The class of homogeneous func- 
tions contains the quadratics as a subclass and is therefore richer than the quadratics. 
Functions which have a singular matrix of second partials at the minimum can be more 
accurately approximated by the homogeneous model. Jacobson and Oksman report that 
their algorithm is competitive with the DFP and CG; however, as originally formulated, 
the JO algorithm is a serial technique. In fact, it was the serial nature of the then -current 
computers that would appear to have influenced Jacobson and Oksman to form their algo- 
rithm as they did. 

The purpose of this report is to describe an algorithm based on a homogeneous 
function model which exploits vector -streaming or parallel -computing capabilities. The 
algorithm is called the parallel Jacobson -Oksman (PJO) method. The term parallel was 
chosen to describe the synchronous or simultaneous evaluation of the function and its 
gradient at a number of different points. In addition to describing the algorithm and a 
modification, some conditions guaranteeing convergence are presented. Also presented 
are the results of numerical experiments involving the application of these methods to a 
collection of test functions. 


SYMBOLS 

A positive definite (n x n) symmetric matrix 

A"1 inverse of A 

b n vector of constants 

C (n + 2) x (n + 2) matrix defined in equation (4) 
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elementary vector 


f o 

f(x) 

g(x) 

U,k 

n 

R n 

s k 

t 

x 

x3 


a 

0 

0 

r 

y 


x,\ 


k 


fixed scalar 

function of n variables 
gradient vector of f 
integers 
index 

vector space of n-tuples 
kth search direction 
scalar 

variable, n vector 

jth component of n -vector x 

intermediate iterates (n-vectors) 

n + 2 vector with components Vj (j = 1, 2, . . n + 2) 
scaler defined by Xj^ g(x^ 



n-vector, location of minimum 
n -vector estimate of £ 
scalar, degree of homogeneity 
scalar, estimate of y 
scalars, step size 
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o'! } CTg , . . ., a n+ i set of n vectors which spans R n 

scalar (0 < r < 1) 

= y f(j8) 

estimate of co 
transpose 

norm of n vector; for example |j x|| = Vx^x 

HOMOGENEOUS FUNCTIONS AND THE PJO ALGORITHM 

Let f be a function of the n -tuple x. The function f is said to be homogeneous 
of degree y about the point /3 if for all x throughout a region R containing /3 

f(tx + 0) - i($) = t y [f(x + 0) - f(/3)] (1) 

for all t such that txe R. 

Notice that quadratic functions are homogeneous of degree two about their minimum 
point. This relationship follows because if 

f(x) = f Q + b T x + i x T Ax 

for A an n x n positive definite symmetric matrix, b a fixed vector in R n , f Q 
a scalar, the gradient is g(x) = b + Ax and the minimum is at j3 = -A - lb. As a 

result, f(tx + /3) = f 0 - r b T A _1 b + \ t 2 x T Ax, and thus 

2 Ci 

f(tx + j8) - f(/3) = | t 2 x T Ax = t 2 |f(x + 0) - f(/3)j 

This definition is a generalization of the usual one where /3 = 0 and f(/3) = 0. If 
f is differentiable with gradient g(x), homogeneous of degree y about /3, then Euler’s 
theorem states that for all x s R 

f(/3) = f(x) + i(/3 - x) T g(x) ( 2 ) 

This relationship is established by differentiating equation (1) with respect to t, setting 
t = 1, and replacing x by x - /3. 
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The preceding definition and model equations are as given by Jacobson and Oksman. 
(See ref. 1.) The model equation (eq. (2)) for a homogeneous function forms the basis of 
the PJO algorithm. For each x e R n , equation (2) contains n + 2 constants. These con- 
stants are /3, the location of the minimum, the minimum value of f and y, and the degree 
of homogeneity. Moreover, if y f(/3) is denoted by u> equation (2) can be rewritten as 

U) = y f(x) + (/3 - x) T g(x) (3) 

which is linear in the n + 2 unknowns, y, and the n-tuple /3. This observation is, 
of course, the same observation that led Jacobson and Oksman to their algorithm. How- 
ever, their algorithm is a serial algorithm as each evaluation of equation (3) results in an 
estimate of j3, y, and cv. The next location for evaluation of equation (3) is determined 
by this estimate of /3. Their formulation of the solution of equation (3) at n + 2 points 
was based on the sequential or serial nature of the then -present -day computers. A short 
description of their method follows the description of the new algorithm. 

Most of the computer time required to solve significant optimization problems is 
spent evaluating the function and its gradient. The parallel -computing capabilities of 
advanced computers allow the evaluation of the function and its gradient at a number of 
points simultaneously at a cost in time of not much more than a single evaluation. How- 
ever, if a number of simultaneous gradient and function evaluations were made in parallel, 
the original JO algorithm could not use this inexpensive information because after each 
function and gradient evaluation, a decision is made concerning the location of the next 
evaluation. Hence, the algorithm as stated is inherently serial. 

The modification of the JO algorithm presented here involves parallel evaluation of 
the function and its gradient at n + 2 distinct points (that is, values for x), and the 
solution at those points of the resulting linear system of n + 2 equations in n + 2 
unknowns, o>, y, and j3 being defined by equation (3) . This estimate of /3 then defines 
a search direction. The algorithm proceeds in that direction until a point is found at which 
the value of the object function is decreased. The process is repeated until the norm of 
the gradient is sufficiently small. A formal statement of the algorithm is as follows: 

Step 0: Let Xq be the initial estimate of the location of the minimum and compute f^^ 
and g^XQ^. Let OpOg, . • •> a n +l be a set °f n-vectors which spans R n and any 

proper subset which is linearly independent on R n and let k = 0. 

Step 1 : Define 

xj = x k + CTj (j = 1, 2, . . ., n + 1) 
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Evaluate f and g in parallel at Xj for j = 1, 2, . . ., n + 1. Set x n+ g = x k and 
evaluate equation (3) at the n + 2 points Xj and form the linear system 

Ca=v (4) 

where 


a = 

3 T , y, 

•T 

0 being 

an estimate of /3) 

v r 


1 

(j = 

= 1,2,. . ., n + 2) 


9x J 

1 (i =1,2,. 

. n + 2; 

j = 1, 2, . . ., n) 

c = < 

‘N 

<i = 

1, 2, . . . 

, n + 2; j = n + 1) 


-1 

L 

(i = 

1,2,. . . 

, n + 2; j = n + 2) 


Solve equation (4) for /3, y, and co. 

Step 2: Define the search direction s k by s k = $ - x k and evaluate f(/3) and g(/3); 
if ||g(/3)|| is sufficiently small, stop. If not and f(/3) < f(Xj^, then set x k+ j = j3, 
k = k + 1, and return to step 1. Otherwise, choose the step size by minimizing fjk^ + ASj 
with respect to A. Denote the minimizing A by A^. Let x k+ ^ = x k + A k (/3 - x k j, set 
k = k + 1, and return to step 1. 

In the original Jacobson -Oksman algorithm, step 1 is replaced by assuming that at 
step 1 one has a current estimate of C - * and a. The oldest information in C is then 
replaced by the corresponding information about f at x^. This means that one row of 
C has been replaced. The new C - ^ and a are then computed by Householder's for- 
mula. (See ref. 14.) Finally, step 2 is taken and the process repeats as necessary. 

CONVERGENCE RESULTS 

If f is a homogeneous function of n variables with minimum at (3 and 
Op a g, . . ., a n+ i are such that the system of n + 2 linear equations formed by evalu- 
ating equation (3) at Xq, Xq + a- (j = 1, 2, . . ., n + 1) has a unique solution, then the 
algorithm described herein locates the minimum in one cycle. This is immediate by the 
construction of the algorithm and definition of a homogeneous function. 
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By slight modifications of the algorithm defined, one can obtain more powerful con- 
vergence results. First, it is required that the step direction Sj c as chosen in step 2 
satisfies 



S k 



(5) 


for some scalar r where 0 < r < 1. If this condition is not satisfied, then s^ is 
chosen as -g/x^j. Additionally, it is required that the step size is chosen so that 


f ( x k) - f ( x k + Vk) - 


/g T M 3 

k \ 



g 0 


Then the following stronger convergence result can be established. 

If at every iteration these modifications to the search direction and step size are 
made, the iterates of the PJO algorithm will converge to the location of the minimum of 
a strongly convex function. (A strongly convex function is a function with a 

continuous, uniformly positive definite matrix of second derivatives.) This result follows 
from a more general result given by Ortega and Rheinboldt. (See ref. 14.) 

As will be seen in a later section, in practice the modified algorithm, for which the 
stronger convergence result held, did not perform better than the basic algorithm. In 
fact, the performance of the modified algorithm was sometimes worse in terms of function 
evaluations required to solve the problem . This behavior of minimization algorithms has 
been observed by others (for example, Daniel, ref. 15). 


NUMERICAL CONSIDERATIONS 

In order to insure that the elements of the C matrix were approximately of the 
same magnitude, the n + 2 column was scaled by 0.5f^x^). This scaling meant that the 
linear equation solver returned 2o>/f^Xj^ instead of o>. Since cu was not used in any 
part of the algorithm, this condition was of no consequence. Recall that to locate the 
minimum of a homogeneous function, the vectors • • ■> a n+ i required by step 1 

of the algorithm need only span R n and generate a unique solution to equation (4); how- 
ever, to be more likely that equation (4) has a unique solution, the a vectors were 
chosen so that any subset of n vectors is also linearly independent. In fact, for the 
numerical examples, cr n+ i is chosen as the sum of the first n a vectors. Of interest, 
of course, is the sensitivity of the performance to other criteria for choosing these vec- 
tors of the algorithm when minimizing general functions. Clearly, the length of the vec- 
tors CTi should be small enough to insure that the local representation of f(x) as an 
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homogeneous function is valid; however, this means that the entries of the rows of the 
C matrix of equation (4) are very similar. For example, the n + 1 column of C is 


given by 


f ft)’ f ( X k + 


f ( x k + °n+l) 


Since the norm of a- is small for 


every i, this means that the entries are very close numerically. Hence, for very small 
crj, the system given by equation (4) may not have full rank or may be very close to being 
singular. Therefore, the solution of the system of equations may be sensitive to small 
changes in the entries of the C matrix. Thus, it would seem that there are two factors 
affecting the performance of the algorithm: (1) the criteria for choosing the a vectors 

and (2) the linear solver of equation (4). In order to investigate these factors, two tech- 
niques of solving the linear system and several schemes of choosing a vectors were 
used. The solution techniques used were Gaussian elimination with total pivotal strategy 
(ref. 16) and a singular value decomposition technique (ref. 17). The latter gives the rank 
of the matrix, its singular values, and the least-squares solution to the linear system 
given by equation (4). The latter algorithm is generally considered to give the better 
(that is, more accurate) solution. 


In general, the following observations are made after many numerical experiments 
involving the test functions described: 

(1) The performance of the PJO algorithm was independent of the linear solver used. 
By independent, it is meant that with both linear solvers, convergence to the minimum was 
achieved in essentially the same number of function evaluations for the same problem. 


The use of the singular value decomposition linear solver did indicate the fact that 
the C matrix is not necessarily of full rank. On two example problems, the rank of the 
C matrix was computed as n - 1 for the first iteration; however, as the iterations pro- 
ceeded, C was of full rank and remained so. Table I lists the rank of C, the largest, 
and the smallest singular values for each iteration while minimizing one of the sample 
functions. Note that after the first iteration, the singular values are well within the 
15 -place accuracy of the computers used in the experiments. Although the performances 
were generally independent, in one anomolous case the algorithm using the Gaussian elim- 
ination minimized one of the test functions in 3 cycles and only 19 function evaluations, 
whereas this function usually required at least 11 cycles and about 67 function evaluations. 
This one case of phenomenal performance is due to a fortuitous first cycle which, while 
decreasing the function value from 215 to only 214, gave rise to a good starting point for 
the second cycle. The second cycle then decreased the function to 10~5, and the final 
step to 10“ 10. Ordinarily, the decrease in the first step is from 215 to about 150. Then 
the algorithm proceeds to decrease f(x) by about an order of magnitude per cycle. 
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(2) The experiments indicated that the manner in which the CTj vectors were cho- 
sen was more important in terms of overall performance than the linear equation -solving 
technique. 

The following schemes referred to as methods (a) and (b) for choosing the vec- 
tors were investigated: 

(a) The first method (method (a)) for choosing the Oj vectors was to let cr^ = e^c 

T 

for i = 1, 2, . . n and a n+ j = c{l, 1, . . ., l} where e^ denotes the ith elementary 
vector and c is a scalar fixed for all iterations. Figure 1 illustrates the performance 
of the algorithm in terms of function evaluations on three sample problems, for c = 10 - 2, 
10” , 10” , 10 , and 10”°. Notice that as c decreases, the number of function evalua- 

tions required for convergence decreases until a minimum is reached; thereafter, the 
performance deteriorates rapidly. The term robustness is used herein to describe the 
relative insensitivity of the algorithm to the magnitude of llc^ II and the techniques of 
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C\ UD Banana 
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Figure 1.- Robustness of PJO (a vectors chosen by method (a)). Symbol with 
asterisk denotes that function did not converge in 500 function and gradient 
evaluations; Powell's function for norm of c r = 10”® was not tested. 
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choosing c^. For the test problems solved herein, it was observed that PJO had about 
a 2 to 3 order of magnitude deadband on performance in terms of function evaluations 
required to converge to the solution. 

(b) The second technique (method (b)) for choosing the vectors was to let 

<x = -cei times sgn ^ith component of for i = 1, 2, . . n and °n+l - E °i 

i=l 

^ith component of g where c is a scalar. This technique seemed to improve the 
overall performance of the algorithm, but not to a great degree. The improvement was 
probably due to the fact that the approximation to f by a homogeneous form was local- 
ized and coordinated with the probable n -dimensional orthant of decrease. (An orthant 
is an n-dimensional quadrant.) 


A final numerical consideration was that the one -dimensional minimization required 
in step 3 of the algorithm was carried out as follows. The initial step -size estimate is 
given by 




rT 

»n 



where is the estimate of X at the kth cycle and f m 


is the estimated value of f 
at the minimum. If ffx^ + y^SjJ < f(x^, the next cycle is begun; otherwise, Davidon's 


cubic interpolation (eq. (3)) is performed to locate the one -dimensional minimum. 


EXAMPLE PROBLEMS AND RESULTS 


To illustrate the performance of the minimization algorithms defined herein, numer- 
ical experiments were conducted on several standard example problems. Although the 
experiments were conducted in single precision on a Control Data Corporation (CDC) 6000 
series computer, the results of the computations were used as if they had been done in a 
parallel fashion. Five sample functions were used for the numerical experiments. 

The first function was a simple quadratic function of three variables 
f^(x,y,z) = x^ + 2y2 + 5z2 - 2xy 

As indicated in the section "Convergence Results,” convergence was achieved in one cycle. 
The second function was Rosenbrock's parabolic valley function 

2 

f 2 (x,y) = (x 2 - y) + 0.01(x - l) 2 
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This is a particularly difficult function to minimize because of the long parabolic valley 
y = x^ along which the minimization must travel. The traditional starting point is 
(-1.2, 1.0) and the minimum is located at (1,1). 

The third function known as Powell's function is 

f 3( x l’ x 2’ x 3’ x 4) = ( X 1 - 10x 2) + 5 ( X 3 ’ X 4) + ( x 2 ~ 2x s) + 10 ( x l ~ x 4) 

with starting values at (3, -1,0,1). This function is particularly difficult for a variable 
metric algorithm to minimize because at the minimum the Hessian is singular. The algo- 
rithm described herein performed extremely well on this problem. 

The fourth test function is called the 4-D Banana and is defined by 



The traditional starting estimate is at (-3,-1, -3,-1). This function is difficult to minimize 

2 2 

because the quadratics x^ - Xg and Xg - x^ make the level surfaces banana shaped. 

The fifth function is the helical valley function defined by 


f 5( x l’ x 2’ x 3) = 100 

where 

r 1 

tan -1 Xg^x. 

2t t9 = < 

7 r + tan - '*' x 

The usual starting estimate is at (-1,0,0). 

The key factor in the performance of any minimization technique is the number of 
function evaluations required for convergence as the computation required for evaluating 
the function is usually much greater than that involved in the algorithm. It is for this 
reason that the results of the experiments are given in terms of the number of function 
evaluations required to achieve convergence. 

Table H gives some of the results of these numerical experiments. For these 
experiments the convergence criteria was that the largest component of the gradient be 
less in magnitude than e = 10"®. The a vectors were defined by method (b) with 


x 3 - 100) + ll/xj + x\ - 1 


+ x, 


Xj ^0 


a/ x i ( : 


x^ < 0 
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e = 10" 4 , that is, = (l0~ 4 e^ ^sgn ith component of g(x k )j and a n+1 = -lO^e^sgn ith 

component of g^J x 10 ” 4 ) where ej is the ith elementary vector. For each test func - 
tion the number of cycles required to achieve convergence is listed. Also listed is the 
total number of function and gradient evaluations required for convergence on the serial 
computer and on an ideal parallel computer. By assuming that the computations had been 
done on an ideal parallel computer with p processors (pln+ 1) , step 1 would have 
been performed by utilizing these capabilities; therefore, if overhead costs are ignored 
and an ideal parallel computer is assumed, these gradient and function evaluations would 
take essentially the same time as one such computation. It is for this reason that the 
entry in column (jf) is found by taking the number of variables times the number of cycles 
and subtracting the result from the entry in column ( 2 ). Columns (T) and (5) give meas- 
ures of the accuracy of the minimization. 

Table III shows the same information for the modified algorithm for which the 
results given in "Convergence Results" hold with r = 0.001. A comparison of tables II 
and III shows that the modification which guarantees convergence does not help the per- 
formance of the algorithm to any significant degree. 

Table IV shows the results for the modified PJO with the a vectors chosen by 
method (a) given and with c = 10 " 4 . A comparison of tables III and IV shows that the 
performance of the modified algorithm is not radically affected by either of the two ways 
of choosing the a vectors. The same situation held for the PJO itself. 

Table V gives the performance of the algorithm by use of the singular value decom- 
position linear solver. Note here again that the performance of the algorithm is not 
radically affected. 

Table VI illustrates a comparison of the performance of the DFP algorithm with the 
PJO algorithm on the four example functions, f2, fg, and f^. The DFP method 
was chosen as the standard of comparison because of its wide use and generally favorable 
comparison with other techniques. Table VI lists the number of function evaluations 
required by the DFP method to locate the minimum starting from the same initial condi- 
tions. These results were reported by Jacobson and Oksman (ref. 13). Finally, in 
table VI, the performance of the PJO is given for two cases. In the first case, the method 
is used on a serial computer; hence the operations in step one (that is, gradient evalua- 
tions) are not done in parallel. For the second case it is assumed that the operations in 
step 1 are carried out in parallel. Hence the n + 1 gradient and function evaluations of 
step 1 will require only the time to carry out one evaluation. So the entry in table V for 
the parallel case is merely the same as that for the serial case minus n times the num- 
ber of cycles. The next two entries give the performance of the PVM algorithm for these 
same problems. 
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CONCLUDING REMARKS 

The numerical experiments indicate that the parallel Jacobson-Oksman (PJO) algo- 
rithm outlined herein can be used to exploit effectively parallel computing capabilities to 
solve unconstrained optimization problems in less computation time. In fact, the per- 
formance of the algorithm indicates that for small problems (that is, two or three vari- 
ables) the PJO can be competitive with currently used serial algorithms. Also, in the 
parallel mode the PJO appears to be competitive with another proposed parallel optimi- 
zation algorithm, the parallel variable metric algorithm. 

It is interesting that the modified PJO did not perform as well on the sample prob- 
lems as did the PJO, in spite of the more general convergence theory for the modified 
algorithm; however, this phenomenon is not uncommon. A note of concern about the PJO 
is the limited robustness of the algorithm; however, the 2 to 3 order of magnitude robust- 
ness band was encouraging. 

Langley Research Center 

National Aeronautics and Space Administration 
Hampton, Va. 23665 
July 8, 1975 
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TABLE I.- TYPICAL RANK, SINGULAR VALUE RANGE OF C MATRIX 




Iteration 

Computed rank, 
full rank = 5 

Largest 
singular value 

Smallest 
singular value 

1 

4 

6.9 x 103 

0 


2 

5 

81 

2.5 

x 10 -8 

3 

5 

115 

7.7 

x 10~ 10 

4 

5 

80.3 

4.8 

x lO" 9 

5 

5 

67.4 

2.7 

x 10 ' 9 

6 

5 

34.3 

1.5 

x 10' 9 

7 

5 

60.9 

1.8 

x 10~ 8 

8 

5 

20.7 

6.5 

x 10‘ 9 

9 

5 

18.7 

7.3 

x 10~ 9 

10 

5 

16.4 

8.3 

x lO' 9 

11 

5 

14.6 

9.2 

X 

t—l 

o 

i 

CD 

12 

5 

13.8 

10 

x lO* 8 

13 

5 

11.8 

1 

x 10-8 

14 

5 

8 

9 

x lO" 9 

15 

5 

4.3 

7.7 

x lO" 9 

16 

5 

1.2 

5.5 

x 10~ 9 

17 

5 

5.9 x 10~ 2 

4.6 

x 10' 9 

18 

5 

1.2 x lO- 2 

2.46 xlO" 11 J 


TABLE U.- RESULTS OF NUMERICAL EXPERIMENTS FOR THE 
PARALLEL JACOBSON -OKSMAN ALGORITHM (PJO) 

[Experimental conditions: chosen by method (b); c = 10 _4 J 



® 

® 


© 

© 

Function 

Number of 
cycles 

Total evaluation 
(serial) 

Total evaluation 
(parallel) 

Function value 
at convergence 

Norm of 
gradient 

Quadratic 

1 

6 

3 

6.6 x 10" 20 

9.7 x 10-19 

Rosenbrock 

20 

87 

47 

9 x 10~ 14 

1.6 x lO’ 6 

Powell 

3 

19 

7 

1.9 x lO* 18 

1.3 x 10‘ 6 

4-D Banana 

36 

227 

87 

6.6 x 10“ 12 

3 x 10~ 8 

Helical valley 

14 

S . 

74 

32 

5.9 x lO’ 18 

5 x 10‘ 8 
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TABLE HI. - RESULTS OF NUMERICAL EXPERIMENTS FOR THE MODIFIED 
PARALLEL JACOBSON -OKSMAN ALGORITHM 

[Experimental conditions: a. chosen by method (b); c = 10 -4 ; T = o.OOlj 


Function 

Number of 
cycles 

Total evaluation 
(serial) 

Total evaluation 
(parallel) 

Function value 
at convergence 

Norm of 
gradient 

Rosenbrock 

20 

87 

47 

9 x 10" 14 

5 x 10- 8 

Powell 

13 


28 

2 x 10‘ 9 

3 x 10' 6 

4-D Banana 

34 

223 

87 

7 x 10' 13 

4 x 10" 6 

Helical valley J 

14 

1 

74 

32 

3 x 10" 16 

5 x IQ" 7 


TABLE IV.- RESULTS OF NUMERICAL EXPERIMENTS FOR THE MODIFIED 
PARALLEL JACOBSON -OKSMAN ALGORITHM 

[Experimental conditions: cl chosen by method (a) ; c = 10“ 4 ; r= O.OOll 


Function 

Number of 
cycles 

Total evaluation 
(serial) 

Total evaluation 
(parallel) 

Function value 
at convergence 

Norm of 
gradient 

Rosenbrock 

21 

91 

49 

4.5 X 10-15 

3 x 10“ 8 

Powell 

11 

67 

23 

1.98 x 10 -9 

4 x 10‘ 6 

4-D Banana 

33 

208 

76 

7 xlO' 14 

2.9 x 10‘ 6 

Helical valley 

18 I 

92 

38 

3.6 x 10 -14 

3.5 x 10" 6 









TABLE V.- PERFORMANCE OF PJO WITH SINGULAR VALUE 


DECOMPOSITION LINEAR SOLVER 


^Experimental conditions: cl chosen by method (b); c = 10~^j 


Function 


Number of 
cycles 


Rosenbrock 


20 


Total evaluation 
(serial) 

87 


Total evaluation 
(parallel) 

47 


Powell 


11 


67 


23 


4-D Banana 


34 


223 


76 


Helical valley 


14 


74 


32 


TABLE VI.- RESULTS OF NUMERICAL EXPERIMENTS: TOTAL NUMBER OF 
FUNCTION AND GRADIENT EVALUATIONS REQUIRED TO ACHIEVE 
CONVERGENCE FOR EXAMPLE FUNCTIONS 



Quadratic 

function 

Rosenbrock’ s 
function 

Powell’s 

function 

4-D 

Banana 

Helical 

valley 

Parallel 

Jacobson -Oksman 
(Serial computer) 

6 

74 

19 

204 

74 

Parallel 

Jacobson -Oksman 
(Parallel computer) 

3 

40 

7 

76 

32 

Davidon -Fletcher -Powell* 
(Serial computer) 

Not reported 

165 

1 80 

161 

76 

PVM 

(Serial computer) 

7 

58 

118 

185 

75 

PVM 

(Parallel computer) 

5 

45 

61 

95 

47 


* 

As reported by Jacobson and Oksman (ref. 13). 
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